The Annals of the American Academy Research Design and Study Outcomes

Does the type of research design used in a crime and justice study influence its conclusions? Scholars agree in theory that randomized experimental studies have higher internal validity than do nonrandomized studies. But there is not consensus regarding the costs of using nonrandomized studies in coming to conclusions regarding criminal justice interventions. To examine these issues, the authors look at the relationship between research design and study outcomes in a broad review of research evidence on crime and justice commissioned by the National Institute of Justice. Their findings suggest that design does have a systematic effect on outcomes in criminal justice studies. The weaker a design, indicated by internal validity, the more likely a study is to report a result in favor of treatment and the less likely it is to report a harmful effect of treatment. Even when comparing randomized studies with strong quasi-experimental research designs, systematic and statistically significant differences are observed. ANNALS, AAPSS, 578, November 2001 50 David Weisburd is a senior research fellow in the Department of Criminology and Criminal Justice at the University of Maryland and a professor of criminology at the Hebrew University Law School in Jerusalem. Cynthia M. Lum is a doctoral student in the Department of Criminology and Criminal Justice at the University of Maryland. Anthony Petrosino is a research fellow at the Center for Evaluation, Initiative for Children Program at the American Academy of Arts and Sciences and a research associate at Harvard University. He is also the coordinator of the Campbell Crime and Justice Coordinating Group. NOTE: We are indebted to a number of colleagues for helpful comments in preparing this article. We especially want to thank Iain Chalmers, John Eck, David Farrington, Denise Gottfredson, Doris MacKenzie, Joan McCord, Lawrence Sherman, Brandon Welsh, Charles Wellford, and